H2HH: Introduction to the Hoffman2 Cluster

Raffaella D'Auria, PhD

Learning Outcomes

• what is the Hoffman2 Cluster
• what can the Hoffman2 Cluster do for you
• navigating the documentation / policies
• getting an account on Hoffman2
• connecting to the Hoffman2 Cluster and opening a graphical application
• unix command line 101
• where to look for applications already available on Hoffman2

Nomenclature

Compute node:

• A compute node is a server (computer) with multiple computing cores that can execute as many simultaneous processes or threads as many cores it has

Computing core:

• A basic unit of computation within a CPU

Login node:

• A server designed to ensure connectivity between the "world" and the computing nodes on the cluster

Nomenclature (cont'd)

Computing Cluster:

• A computing cluster is made of individual nodes (computers) which are interconnected between themselves and which aggregate power can be harnessed to address problems which nature requires either distributed or capacity computing

What can the Hoffman2 Cluster do for you

High Performance Computing

• distributed computing (multiple computing cores across multiple compute nodes)
• shared-memory computing (multiple cores on the same compute node)
• data-intensive computing
• high-memory computing

Capacity Computing

• multiple computing cores available to carry out very many simulations
• execute concurrent (synchronous) or indeopendent (asynchronous) simulations

What can the Hoffman2 Cluster do for you (cont'd)

Licensed software availability

• Abaqus
• Intel Parallel Cluster Studio
• Maple
• Mathematica
• MATLAB

Licensed software management

• bring your own licensed software (COMSOL, Ansys, FDTD, etc.)

User Support/Project Collaboration/Training

Navigating the documentation

https://www.hoffman2.idre.ucla.edu/

The Hoffman2 Cluster - About

https://www.hoffman2.idre.ucla.edu/About/System-overview.html

Getting an account on Hoffman2

https://www.hoffman2.idre.ucla.edu/Accounts/Requesting-an-account.html

Requesting a user account

https://www.hoffman2.idre.ucla.edu/Accounts/Requesting-an-account.html#applying-for-a-user-account

Requesting a sponsor account

https://www.hoffman2.idre.ucla.edu/Accounts/Requesting-an-account.html#register-as-a-sponsor

Sponsor responsibilities

https://www.hoffman2.idre.ucla.edu/Accounts/Sponsors-managing-your-group.html#sponsor-responsibilities

Managing your account

https://www.hoffman2.idre.ucla.edu/Accounts/Users-managing-your-account.html#password-processes

Password Processes

https://www.hoffman2.idre.ucla.edu/Accounts/Users-managing-your-account.html#password-processes

For sponsor: managing users in your group

https://www.hoffman2.idre.ucla.edu/Accounts/Sponsors-managing-your-group.html#managing-your-users

Connecting to the Hoffman2 Cluster

https://www.hoffman2.idre.ucla.edu/Using-H2/Connecting/Connecting.html

Connecting via a terminal and an SSH client

https://www.hoffman2.idre.ucla.edu/Using-H2/Connecting/Connecting.html#connecting-via-terminal-and-ssh

Terminal and SSH client (Linux and MacOS)

Terminal and SSH client (Linux and MacOS) (cont'd)

Terminal and SSH client (Linux and MacOS) (cont'd)

Connecting via a terminal and SSH for the first time

Connecting for the first time

https://www.hoffman2.idre.ucla.edu/About/FAQ/FAQ.html#connecting-for-the-first-time

Hoffman2 Cluster public hostkey fingerprints

https://www.hoffman2.idre.ucla.edu/About/FAQ/FAQ.html#table-h2-public-host-classes

Connecting from Windows with terminal and SSH

https://www.hoffman2.idre.ucla.edu/Using-H2/Connecting/Connecting.html#connecting-via-terminal-and-ssh

On Windows one of the following applications should be installed:

• MobaXterm terminal for Windows with X11 server, SSH client & network tools
• GitBash terminal with SSH client
• Windows Subsystem A Windows native compatibility layer for running Linux binaries on Windows (available from Windows 10)
• PuTTY terminal with SSH client
• Cygwin a large collection of GNU and Open Source tools which provide functionality similar to a Linux distribution on Windows

X Window Sytems/X11

The software that allow displaying of graphical applications on a unix-like system and that is designed to work on a server-client mode (you can connect to it and forward graphical applications to your local computer)

• For example: forward the MATLAB desktop GUI from Hoffman2 to your local computer

Opening graphical applications on Linux (Hands-on)

• X11 server is natively available
• forwarding of graphcail applications for remote display is enabled via:

  ssh -X joebruin@hoffman2.idre.ucla.edu

  substitute joebruin with your Hoffman2 username

Opening graphical applications on MacOS (Hands-on)

first time only:

• install Xquartz (the X11 server for MacOS)
• open the terminal app and before connecting to Hoffman2 issue:

defaults write org.macosforge.xquartz.X11 enable_iglx -bool true

• reboot your local computer

connect with:

• forwarding of graphcail applications for remote display is enabled via:

  ssh -Y joebruin@hoffman2.idre.ucla.edu

  substitute joebruin with your Hoffman2 username

Opening graphical applications on Windows

no need to install other software then any of the following:

• MobaXterm
• Windows Subsystem for Linux
• Cygwin

need to install an X11 window server if using either of:

• GitBash
• PuTTY

a possible X11 Window server:

• Xming

Opening graphical applications on Windows using MobaXterm (Hands-on)

Opening graphical applications on Windows using MobaXterm (cont'd)

Opening graphical applications on Windows using MobaXterm (cont'd)

Opening graphical applications on Windows using MobaXterm (cont'd)

Opening graphical applications on Windows using MobaXterm

Unix command line 101

https://www.hoffman2.idre.ucla.edu/Using-H2/Command-line/Unix-command-line.html

Unix command line 101: Navigation (Hands-on)

https://www.hoffman2.idre.ucla.edu/Using-H2/Command-line/Unix-command-line.html#navigation

• Try these commands:

ls
ls -a
ls -l
ls -lat
ls -latr
pwd
cd $SCRATCH
pwd
timestamp=`date "+%F_%H-%M"`; mkdir $HOME/H2HH_$timestamp; cd $HOME/H2HH_$timestamp
pwd
rmdir $HOME/H2HH_$timestamp
ls $HOME/H2HH_$timestamp

Move to Hands-on notebook

Unix command line 101: Environmental Variables (Hands-on)

https://www.hoffman2.idre.ucla.edu/Using-H2/Command-line/Unix-command-line.html#unix-environment-environmental-variables

• Some important environmental variables:

$PATH
$LD_LIBRARY_PATH
$HOME
$SCRATCH  # Hoffman2-specific
$SHELL

• to check the content of an environmental variable issue:

echo $HOME

Move to Hands-on notebook

Unix command line 101: Working with files (Hands-on)

https://www.hoffman2.idre.ucla.edu/Using-H2/Command-line/Unix-command-line.html#working-with-files

• Try these commands:

  cat $HOME/.bashrc
  less $HOME/.bashrc   # type q to interrupt
  more $HOME/.bashrc   # type q to interrupt

• Check the first few lines of a file:

  head $HOME/.bashrc
  head -n 2 $HOME/.bashrc

• Check the last few lines of a file:

  tail $HOME/.bashrc
  tail -n 2 $HOME/.bashrc

• Check the last lines of a file as it is being written:

  tail -f $SGE_ROOT/$SGE_CELL/common/accounting   # Control-C to interrupt

Move to Hands-on notebook

Unix command line 101: Editing files (Hands-on)

https://www.hoffman2.idre.ucla.edu/Using-H2/Command-line/Unix-command-line.html#editing-files

• Non graphical editors:

  nano 
  emacs
  vi

• Graphical editors:

  gedit &
  emacs &
  gvim &

Getting work done on the cluster

On which resources will your job run?

https://www.hoffman2.idre.ucla.edu/Using-H2/Computing/Computing.html#computational-resources-on-the-hoffman2-cluster

Highp vs shared vs campus resources:

https://www.hoffman2.idre.ucla.edu/Using-H2/Computing/Computing.html#highp-vs-shared-vs-campus-jobs

• highp refers to the use of group owned compute nodes
  • users can run jobs for up to 14 days
  • only for users in group who own resources
• shared refers to the use of temporarely unused group owned compute nodes
  • users can run jobs for up to 24 hours
  • only for users in group who own resources
• campus refers to compute nodes owned by IDRE and made available to the UCLA community
  • users can run jobs for up to 24 hours

On which resources will your job run? (cont'd)

https://www.hoffman2.idre.ucla.edu/Using-H2/Computing/Computing.html#jobs-and-resources

Working interactively on the cluster

Where to look for applications already available on Hoffman2

Running non interacive workflow (batch jobs)